Add unified jobs API with /api/jobs endpoints #11054

ric-yu · 2025-12-02T19:23:42Z

Summary

Add GET /api/jobs endpoint with filtering, sorting, and pagination
- Filter by status: pending, in_progress, completed, error
- Sort by create_time or execution_time
- Pagination via limit/offset
Add GET /api/jobs/{job_id} for single job details with full outputs
Track execution_time in history for completed jobs
Include outputs_count and preview_output in job responses
Preview logic matches frontend (prefers type=output, supports images/video/audio/3D)

Motivation

Currently /queue and /history are separate endpoints with different response shapes. This unified API provides:

Consistent job representation across all states
Proper pagination and filtering
Summary fields (outputs_count, preview_output) for list views without fetching full outputs

Test plan

Run existing tests: pytest tests/execution/test_execution.py -v
Verify /api/jobs returns jobs from queue and history on https://www.github.com/Comfy-Org/ComfyUI_frontend/commit/a57ea57a09a274efdc8444b949452509d7cf49de
Verify /api/jobs/{id} returns full job with outputs
Verify execution_time is tracked for completed jobs

luke-mino-altherr · 2025-12-02T20:22:04Z

execution.py

likely out of scope and should be done in separate pass, but can we break up the routes in this file to separate files. Over 1000 lines is bananas

luke-mino-altherr · 2025-12-02T20:24:37Z

execution.py

+    def _normalize_queue_item(self, item, status):
+        """Convert queue item tuple to unified job dict."""
+        _, prompt_id, _, extra_data, _ = item[:5]
+        create_time = extra_data.get('create_time')
+
+        return {
+            'id': prompt_id,
+            'status': status,
+            'create_time': create_time,
+            'execution_time': None,
+            'error_message': None,
+            'outputs_count': 0,
+            'preview_output': None,
+            'workflow_id': None,
+        }
+
+    def _normalize_history_item(self, prompt_id, history_item, include_outputs=False):
+        """Convert history item dict to unified job dict."""
+        prompt_tuple = history_item['prompt']
+        _, _, prompt, extra_data, outputs_to_execute = prompt_tuple[:5]
+        create_time = extra_data.get('create_time')
+
+        # Determine status from history status
+        status_info = history_item.get('status', {})
+        if status_info:
+            status = 'completed' if status_info.get('status_str') == 'success' else 'error'
+        else:
+            status = 'completed'
+
+        outputs = history_item.get('outputs', {})
+
+        outputs_count, preview_output = self._get_outputs_summary(outputs)
+
+        error_message = None
+        if status == 'error' and status_info:
+            messages = status_info.get('messages', [])
+            if messages:
+                error_message = messages[0] if isinstance(messages[0], str) else str(messages[0])
+
+        execution_time = history_item.get('execution_time')
+
+        job = {
+            'id': prompt_id,
+            'status': status,
+            'create_time': create_time,
+            'execution_time': execution_time,
+            'error_message': error_message,
+            'outputs_count': outputs_count,
+            'preview_output': preview_output,
+            'workflow_id': None,
+        }
+
+        if include_outputs:
+            job['outputs'] = outputs
+            job['prompt'] = prompt
+            job['extra_data'] = extra_data
+            job['outputs_to_execute'] = outputs_to_execute
+
+        return job


a lot of these private methods appear to be static methods. To keep this class thinner, can we break out the static methods to a separate file, and move outside the class to be regular functions?

luke-mino-altherr · 2025-12-02T20:29:19Z

server.py

similar comment to before -- likely out of scope for this, but can we start introducing some structure to this repo and start breaking up this superclass to separate files for maintainability and extensibility

luke-mino-altherr · 2025-12-02T20:33:06Z

execution.py

+            jobs = []
+
+            if status_filter is None:
+                status_filter = ['pending', 'in_progress', 'completed', 'error']


Are these statuses defined elsewhere in this file? Worth breaking out to an enum to ensure consistency within this file?

luke-mino-altherr · 2025-12-02T20:37:40Z

execution.py

+                    if (is_error and include_error) or (not is_error and include_completed):
+                        jobs.append(self._normalize_history_item(prompt_id, history_item))
+
+            jobs = self._apply_sorting(jobs, sort_by, sort_order)


nit: apply sorting after limiting

we gotta sort first or we'll limit the wrong entries :(

comfyanonymous · 2025-12-02T22:27:25Z

execution.py

+                        media_type == 'video' or
+                        media_type == 'audio' or
+                        filename.endswith(('.png', '.jpg', '.jpeg', '.gif', '.webp', '.bmp')) or  # images
+                        filename.endswith(('.mp4', '.webm', '.mov', '.avi')) or  # video


This is a bad way to do this.

I can refactor this a bit, but the frontend counts by checking file extensions and skipping animated

execution.py

ric-yu · 2025-12-04T20:53:28Z

Rebasing

bigcat88 · 2025-12-05T03:36:11Z

comfy_execution/jobs.py

+    IN_PROGRESS = 'in_progress'
+    COMPLETED = 'completed'
+    FAILED = 'failed'
+    CANCELLED = 'cancelled'


Should the cancelled status exist?

The /api/jobs endpoint will always return an empty list for this status, and normalize_history_item will treat this status as JobStatus.COMPLETED.

bigcat88 · 2025-12-05T03:44:42Z

server.py

+            status_param = query.get("status", None)
+            status_filter = None
+            if status_param:
+                status_filter = [s.strip() for s in status_param.split(',') if s.strip()]


If the caller passes status=foo, then:

status_filter becomes ["foo"].

Filtered against valid_statuses -> []

status_filter becomes None and call returns full results with 200 OK

That will be pretty surprising for the caller

bigcat88 · 2025-12-05T03:47:24Z

server.py

+                    status=400
+                )
+
+            sort_order = query.get('sort_order', 'desc')


Is it possible to add lowercase conversion here?

comfyanonymous · 2025-12-05T04:19:15Z

Can you test this properly? The endpoint doesn't work after I run a regular workflow.

ric-yu requested a review from guill as a code owner December 2, 2025 19:23

ric-yu marked this pull request as draft December 2, 2025 19:24

ric-yu force-pushed the feature/unified-jobs-api branch 2 times, most recently from ae57e94 to c4b7a84 Compare December 2, 2025 19:52

ric-yu marked this pull request as ready for review December 2, 2025 20:03

ric-yu requested review from Kosinkadink and comfyanonymous as code owners December 2, 2025 20:03

luke-mino-altherr reviewed Dec 2, 2025

View reviewed changes

comfyanonymous reviewed Dec 2, 2025

View reviewed changes

Kosinkadink added the Core Core team dependency label Dec 3, 2025

comfyanonymous reviewed Dec 3, 2025

View reviewed changes

execution.py Outdated Show resolved Hide resolved

ric-yu marked this pull request as draft December 4, 2025 00:50

ric-yu marked this pull request as ready for review December 4, 2025 05:31

ric-yu marked this pull request as draft December 4, 2025 05:31

ric-yu marked this pull request as ready for review December 4, 2025 07:52

ric-yu added 11 commits December 4, 2025 12:53

feat: create a /jobs api to return queue and history jobs

5aa07d7

update unused vars

aa00641

include priority

b2bd48e

create jobs helper file

b874f46

fix ruff

380b6ae

update how we set error message

048c413

include execution error in both responses

2e0b26b

rename error -> failed, fix output shape

e4c7136

re-use queue and history functions

90fb5cc

set workflow id

8a2bb7c

allow srot by exec duration

b7c7712

ric-yu added 4 commits December 4, 2025 12:53

fix tests

a38aacf

send priority and remove error msg

1b67306

use ws messages to get start and end times

5274b91

revert main.py fully

c860cc6

ric-yu force-pushed the feature/unified-jobs-api branch from 57fd19f to c860cc6 Compare December 4, 2025 20:53

ric-yu added 3 commits December 4, 2025 13:36

refactor: move all /jobs business logic to jobs.py

5ed90a1

fix failing test

b14ae80

remove some tests

a10cb30

bigcat88 reviewed Dec 5, 2025

View reviewed changes

Add unified jobs API with /api/jobs endpoints #11054

Are you sure you want to change the base?

Add unified jobs API with /api/jobs endpoints #11054

Conversation

ric-yu commented Dec 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Motivation

Test plan

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ric-yu commented Dec 4, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bigcat88 Dec 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

comfyanonymous commented Dec 5, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

ric-yu commented Dec 2, 2025 •

edited

Loading

bigcat88 Dec 5, 2025 •

edited

Loading